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Abstract — Asymptotic concentration behaviors of linear com- 
binations of weight distributions on the random linear code 
ensemble are presented. Many important properties of a binary 
linear code can be expressed as the form of a linear combination 
of weight distributions such as number of codewords, undetected 
error probability and upper bound on the maximum likelihood 
error probability. The key in this analysis is the covariance 
formula of weight distributions of the random linear code 
ensemble, which reveals the second-order statistics of a linear 
function of the weight distributions. Based on the covariance 
formula, several expressions of the asymptotic concentration rate, 
which indicate the speed of convergence to the average, are 
derived. 



I. Introduction 

For a binary random code ensemble or a binary random 
linear code ensemble, the asymptotic behaviors of the first 
moment (expectation) of some properties of interest have 
been studied extensively. For example, the error exponent 
derived by Gallager [1] is a celebrated consequence of such 
a first-moment analysis. Recent advances in second-moment 
analysis on low-density parity check matrix ensembles [5], 
[6] have encouraged studies on the second-order behaviors 
(fluctuation from the average) of the macroscopic properties 
of an ensemble, which had previously attracted little attention. 

In this paper, asymptotic concentration behaviors of linear 
combinations of weight distributions on the random linear 
code ensemble are presented. Many important properties of 
a binary linear code can be expressed as the form of a 
linear combination of weight distributions such as number of 
codewords, undetected error probability and upper bound on 
the maximum likelihood (ML) error probability. The key in 
this analysis is the covariance formula of weight distributions 
of the random linear code ensemble, which reveals the second- 
order statistics of a linear function of the weight distributions. 
Based on the covariance formula, several expressions of the 
asymptotic concentration rate, which indicate the speed of 
convergence to the average, are derived. 

II. Preliminaries 

A. Ensemble, expectation, and covariance 

Let Q be a set of binary mxn matrices where m and n are 
positive integers. Suppose that probability P{H) is assigned 
for each matrix H in Q, where P(H) is a probability mass 
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function defined on Q such that Y^hgQ = ^ an( ^ ^ 

Q,P(H) > 0. The pair {Q,P(H)} can be considered as an 
ensemble of matrices. Although it is an abuse of notation, for 
simplicity, we will not distinguish {Q,P(H)} from Q. 

Let /(•) be a real-valued function defined on Q, which 
can be considered as a random variable. The expectation 
of /(•) with respect to the ensemble Q is defined by 

E S [/] = Y< H zgP{H)f{H). The variance of /(•) is given 
by VARg [/] = E g [f(H) 2 } - E g [f(H)} 2 . In a similar way, 
the covariance between two real-valued functions f(-),g(-) 
defined on Q is given by 



COV g [/, 5 ]=E e [/«?]-E e [/]E e [.g]. 



(1) 



Let {gi (•) , #2 (•) ) • • • ) 9n (•) } be a set of real-valued functions 
defined on Q, and let /(•) be a linear combination of gi(-): 

f( H ) = Yh=i 4>igi{H) for H G g, where <^(j G [l,n]) are 
real values. The notation [a, b] denotes the set of consecutive 
integers from a to b. It is easy to show that the variance of 
/(•) is given by 



VAR g [f] = J2J2' 

1=1 i=l 

e.g., see [7] for details. 



iCOVg[g i)9j } 



(2) 



B. Weight distribution 

The weight distributions {Ai(-), . . . , A„(-)}, which can be 
considered as a set of real-valued functions defined on Q, is 
defined by 



-4. 



I[Hx = Q m ], we[0,n] 



(3) 



for any H G Q, where Z t - n ' w ' ) denotes the set of all binary n- 
tuples with weight w. The function /[•] is the indicator function 
such that I[condition] = 1 if condition is true; otherwise, it 
gives 0. In the present paper, symbol shown in bold, such as 
x, denote column vectors. 

Let C(H) be the binary linear code defined based on H, 
namely, C(H) = {x e F 2 ™ : Hx = O" 1 }, where F 2 denotes 
the binary Galois field. Many properties of C(H) of interest 
can be represented by a linear combination of the weight 
distributions {A w (-)}^ =1 . Let F (•) be such a property of 

C(H), which is expressed as F(H) — J2w=i ^w^-w(H) for 
any H G Q, where <fr w (w G [0,n]) are real values. 
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For example, the undetected error probability of C(H) 
can be expressed as a linear combination of the weight 
distributions of C(H) when it is used as an error detection 
code for a binary symmetric channel (BSC). The expression 
is given by F(H) = YZ=l A w {H)e w {l - e) n ~ w , where e is 
the crossover probability of the BSC. 

In this setting, the property F(-) can be regarded as a 
random variable that takes a real value. It is natural to study 
its statistics such as expectation, variance for a given ensemble 
of binary matrices. 

C. Random linear code ensemble 

In the present paper, we deal with an ensemble of binary 
matrices, which is called the random linear code ensemble. 

Definition 1: The random linear code ensemble Hn,m con- 
tains all binary m x n matrices. Equal probability P{H) = 
l/2 nm is assigned for each matrices in lZ n ,m- □ 
Note that although the random linear code ensemble is actually 
an ensemble of matrices, it is regarded herein as an ensemble 
of binary linear codes. 

The expectation of weight distributions of random ensemble 
is known [2] to be E nn m [A w ] = (^)2^ m for n > 1. The next 
theorem provides a closed formula of the covariance of weight 
distributions over the random linear code ensemble. 

Theorem 1: Assume a random ensemble lZ n . m . The covari- 
ance of A Wl (-) and A W2 (-) is given by 

COVii n , m [A Wl ,A W2 ] 

0, < wi , u> 2 < n, wi ^ w 2 

(l-2-™)2- m Q 0< Wl =u; 2 <n. { > 

(Proof) The proof is given in Appendix. q 
The variance of the weight distributions of the random lin- 
ear code ensemble has already been shown in [4]. Thus, 
the new contribution of this theorem is the case in which 
COV-jz n ( A Wl , A W2 ) = when wi ^ w 2 - This theorem 
implies that the pair of random variables A W1 and A W2 (wi ^ 
u> 2 ) is pairwise independent 

III. Formulas on asymptotic concentration rate 

A. Asymptotic behaviors of expectation 

Definition 2: Let Q n be an ensemble of binary (1 — R)n x n 
matrices. The parameter R, called the design rate, is a real 
value in the range of < R < 1. Suppose that /(■) is a 
real-valued function defined on Q. The asymptotic exponent 
of Eg [/] is given by 



£ = lim -logEg„[/] 



if the limit exists. 

Namely, asymptotically, Eg[f] behaves like Eg[f(H)] = 2 
where the notation a n = b n means that 



(5) 



lim (l/n)loga n = lim (l/n)logfe„. 

n — >oc n — >oo 

In the present paper, a logarithm of base 2 is denoted by log. 

'Note that the set of random variables {Ai,...A n } are not mutually 
independent because ^2w=l ^ w 

(H) > 2 n " m - 1 holds for any instance H 

in TZ n , m - 



In the case of the random linear code ensemble, it has been 
reported [2] that 

lim - logE Kn {1 _ R JA en ] = H(6) - (1 - R), (6) 

n— >oo Tl 

holds for < < 1, where H(-) is the binary entropy 
function defined by H(x) = — x log x — (1 — x) log(l — x). 
The parameter 8 is called the normalized weight. 



B. Asymptotic concentration rate 

As the size of the matrix goes to infinity, the value of 
/(•) is often sharply concentrated around its expectation. The 
asymptotic concentration rate is defined as follows. 

Definition 3: Let Q n be an ensemble of binary (1 — R)n x n 
matrices, where R is a real value in the range of < R < 1. 
For a real- valued function /(■) defined on Q n , the asymptotic 
concentration rate (abbreviated as ACR) of /(•) is defined by 



A Um 1 / VARg„[/] 



(7) 



if the limit exists. q 
The following lemma explains the importance of the asymp- 
totic concentration rate. 

Lemma 1: Let r\ be the asymptotic concentration rate of 
/(•). For any positive real number a, 



lim — log Pr 

n^oo Tl 



f(H) 



E 6n [/] 



^ (1 - a, 1 + a) 



< 



n 



(8) 



holds if Eg n [/] > for any sufficiently large n. 

(Proof) Based on the Chebyshev inequality, the inequality 



Pr 



\f(H)-Eg n \f\\>Cy/VAMgJf\ 



< \ (9) 



holds for any real number c > 0. Suppose that c is given by 

- QEs " [/] (10) 
^VARaJ/] 

where a is a positive real number. From the assumption 
Eg„ [/] > 0, it is easy to verify that c becomes positive. 
Substituting ( fTOb into (0, we have 



Pr[\f(H)-Eg n [f}\>aE Gn [f}}< 



VARgJ/j 
^Eg n [I\ 2 ' 



(11) 



Due to the assumption Fig[f(H)] > 0, the above inequality 
can be rewritten in the following form: 



Pr 



mi 

E e „[/] 



^ (1 - a, 1 + a) 



< 



VARg„[/] 
a 2 E e [/] 2 • 



(12) 



Considering the asymptotic exponent of the above equation, 
we obtain the claim of the lemma. q 
From the asymptotic concentration rate, we can clarify the 
probabilistic convergence behavior of /(•). If rj < holds, 
f(H)/Eg n [f} converges to 1 in probability as n goes to 
infinity. This means that rj < is a sufficient condition of 
the convergence in probability. The asymptotic concentration 
rate indicates the speed of this convergence 
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Example 1: The variance of the weight distributions of the 
random linear code ensemble is given by 

VAR nn ^_ R)n [A 8n ] = (l-2-^ n )2-^ n (?). (13) 
Therefore, the asymptotic exponent of the variance becomes 
lira - log VAR^„ [A Bn ] = H{6) - (1 - R). (14) 

From this exponent, we immediately have the asymptotic 
concentration rate of the weight distribution: 

,. 1, VAR K [^ e „] 
r) = hm -log— r 12 

= ff(0)-(l-ii)-2(ir(0)-(l-.R)) 

= 1-R-H(d). (15) 

Let the minimum root of equation 1 — R — H(6) = be 8gv, 
which is called the relative Gilbert-Varshamov (GV) distance. 
Since 77 < holds in the range 6qv < < 1 — 9qv, 
Ag n (H)/E-ji n n-K)n l-A-On] converges to 1 in probability as 71 
goes to infinity [3]. rj 

C. ACR of a linear combination of weight distributions 

The goal of the present paper is to observe the asymptotic 
behavior of the variance of the linear combination defined in 
( [T6l l of the weight distributions: 



F(H) = J2^ W A W (H). 



(16) 



The next theorem gives the asymptotic concentration rate of 
F(H). 

Theorem 2: Let Q n be an ensemble of binary (1 — R)n x 
71 matrices, which have the following asymptotic first- and 
second-order behaviors: 

E g JAe n } = 2»™+«W>, (17) 
COVg n [Ao in ,Ao 2n ] = 2 n "<^< e *\ (18) 

The asymptotic concentration rate of F(-) defined in ( fToT l is 
given by 



sup sup [<K0 1 )+<l>(0 2 )+ 7(01,02)] 
o<0i<io<e 2 <i 

2 sup 

0<6><1 



H(6) + q(6)}, 



(19) 



where (f)(0) is defined by 



= lim — log <&fl 



n — >oo 71 

(Proof) It is easy to verify that 

lim - logE ff „[F(fO] = sup W) + H{9) + q(9)} 

n->oo 71 O<0<1 



(20) 



holds. Using Eq.©, we have 



lim -logVAR e „[F] 

n — >oo ji 



(21) 



sup sup [0(0i)+0(0a) +7(01,02)]. (22) 
0<Si<l o<e 2 <i 



Substituting d2T]> and into the definition of the ACR, the 
theorem is proven. rj 

The next corollary is a special case of the above theorem 
for the random linear code ensemble. 

Corollary 1: The ACR of F(-) defined in ( fToT l over the 
random linear code ensemble TZ n n-m n is given by 

r]= sup [2(f)(6) + H(9)\ - sup [2(f)(9) + 2H(6)] + 1 - R, 
o<e<i o<e<i 

(23) 

where (f> is given in (|20] |. 

(Proof) In the case of the random ensemble, q(9) is given by 
q(9) = —(1 — R) for < Q < 1. From Theorem Q] we can 
derive the exponent of the covariance 7(^1, 62), which is given 
by 

-00, 9 1 ^ 9 2 



l(Vl,V2) = 



H(9)-(l-R), 9i=e 2 , 



(24) 



where < 61,62 < 1. Plugging these functions into the 
formula in Theorem [2] we obtain the claim of the corollary. 

□ 

Example 2: In this example, we will discuss the num- 
ber of codewords in C(H). Let us define M(H) = 1 + 
2~2w=i A W (H), which is the number of codewords of C(H). 
In this case, we can see that = 1 holds for 1 < w < n. 
The asymptotic exponent of M(H) is given by 



lim — log E-jz [M] 

n—*oo Ti 



= sup [H(6)] - (1 - R) 

0<9<1 

= R. (25) 



From the definition of M(H), we immediately have 
0,0 < 6 < 1. Using Corollary Q] we obtain 

?/ = sup [H(6)\ - sup [2H(6)\ + 1-R 
o<9<i o<e<i 

= -R. 



(26) 



Since R is a positive real number, M(H)/Erji n m [M] con- 
verges to 1 in probability for any R > 0. rj 

In some cases, the asymptotic concentration rate can be 
written in a closed from without an optimization process 
required in Corollary Q] 

Theorem 3: Assume the random linear code ensemble with 
design rate R. Let K\ , K2 be real positive constants that do 
not depend on n. If & w is expressed as $ w = K^K^ 1 " , then 
the ACR of F(H) - £™ =1 <£ W A W (H) is given by 



77 = log 



1 - R. 



(27) 



(K\ + K 2 ) 2 

(Proof) Using Theorem Q] and the binomial theorem, we have 

VAR TCn , (I _ fl)n [F] 

n n 



Wi — l 102 = 1 




n 
w 

2\w t T t - r 2\n—w 



mr(Ki) 
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(1 - 2~ m )2 
(1 - 2- m )2~ r " 



— m\m— Tn> j^2n 



{l-2- m )2- m Ki n .(28) 



Thus, the asymptotic exponent of VAR^ _ [F] is given 
by 

lim - log VAR75, [F] = log (X 2 + iff) - (1 - i£). 

(29) 

In a similar way, E^ {1 _ R)n [F) can be rewritten as follows: 

n 



that ML decoding is used in a decoder. For a binary m x n 
parity check matrix H , the block error probability P e (H) can 
be upper bounded as follows: 



Pe(H) < £Ab(H)I> w , 



u; — 1 

where D is called the Bhattacharya parameter and is defined 

as 

A 



\w=0 V y 



i—m Tsn 



= 2- m (K 1 + K 2 

This leads to the exponent of the expectation: 
1 



(30) 



lim -logE^ n [F]=log(K 1 +K 2 )-(l-R). (31) 

n— >oo Ji 

Substituting the above two equations into the definition of the n 
ACR, we have the claim of the theorem. rj (1 _ H) „[-B] = J5 

Example 3: Assume the binary symmetric channel with 
crossover probability e. The undetected error probability of 
C{H) is given by P V {H) = YZ=i A w {H)e w e n ~ w . In this 
case, the error exponent becomes 



D = 2^(1^7). 

The upper bound is called the Bhattacharya bound [1] and has 
the form of a linear combination of weight distributions. Let 

us define B{H) = YZ=i A W (H)D W . It is expected that the 
statistics of B(H) reflects the asymptotic behavior of actual 
ML probability of an ensemble. 

We first derive the asymptotic expression of the error expo- 
nent of the Bhattacharya bound in the case of the random linear 
code ensemble. The expectation of B(H) has the following 
closed form expression: 



,(1-H)r 



[A W (H)]D« 



IU—1 ^ ' 



lim —E nn [Pu] = 1 - R. 



(32) 



Since = e w e n ^ w has the form stated in Theorem [3] (i.e., 
K\ = e, K2 = 1 — e), we can apply Theorem [3] and obtain r\ = 
log(e 2 + (l — e) 2 ) + l — R. This results suggests the existence of 
the convergence threshold e* for given R such that e* separates 
the concentration regime and the non-concentration regime of 
e. The root of log(e 2 + (1 — e) 2 ) + 1 — R = becomes an upper 
bound of e*. Let e' be the root of the equation log(e 2 + (1 — 
e) 2 ) + 1 — R = 0. Table ^presents some values of e' for 0.1 < 
R < 0.9. When e > e', we have log(e' 2 + (l-e') 2 ) + l--R < 0. 
In such a region, Pu(-) concentrates around its average value 
in the limit as n tends to infinity. 

TABLE I 

Roots of log(e 2 + (1 - e) 2 ) + 1 - R = 



= 2-( 1 -^"(2Ve(l-e) + 1)" - 2'^-^ 
Thus, the error exponent of {1 _ R)n [B] is given by 



lim — logE TCn [B] 

n— >oo 77, »v v 



1-iJ-log (2^(1 -e) + l 



(33) 



This is a part of the error exponent function derived by 
Gallager [1] (see also [3]) in the low-rate regim^l Namely, 
the Bhattacharya bound corresponds to the upper bound due 
to Gallager with the parameter p = 1 [1]. 

In the following, we will examine the asymptotic concen- 
tration rate of the Bhattacharya bound. 

Corollary 2: The ACR of B(H) is given by 



Tj = log 



4e(e-l) + 1 



1 - R. 



(34) 



R 



0.1 

0.2 
0.3 
0.4 
0.5 
0.6 
0.7 
0.8 
0.9 



0.366047 
0.307193 
0.259613 
0.217375 
0.178203 
0.140933 
0.104872 
0.069564 
0.034687 



, (2\/c(T— e) + I) 2 / 
(Proof) By letting K\ = D and K2 = 1 and using Theorem [3] 
we obtain ij = log ((D 2 + 1)/(D + l) 2 ) + 1 - R. Substituting 
D = 2\J e(l — e) into this equation, the corollary is proven. 



□ 



B. Expurgated bound 



□ 

IV. ACR OF THE UPPER BOUND OF ML ERROR 
PROBABILITY 

A. Bhattacharya bound 

In the following discussion, the binary symmetric channel 
with crossover probability e is assumed for simplicity. Assume 



We here consider the expurgated ensemble 2* ni (i_m B , 
which can be obtained from 'R- n ,(i-R)n by expurgating parity 
check matrices with Ag n (H) ^ for < 8 < 9 GVl I—Oqv < 
9 < 1. The asymptotic growth rate of the weight distributions 
is the same for the original and expurgated ensembles when 
&GV < 9 < 1 — 6gv- However, q(6) becomes —00 when 

2 It has been reported that this exponent is asymptotically tight if R x < 
R < Rcrit [3]. 
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< 9 < Ogv, 1 — @gv < < 1 in the case of the expurgated 
ensemble. 

The error exponent of ~Erji* n ( i_ R) „ [B] is given by 
ton z!logE K , n((1 _ [B] 



min {1 - R- H(9) - 0\og(2Je(l - e))}. 

GV <9<1-0 G V 

If ^cri* > Ogv, the minimum in the above equation is attained 



at 



it, where 



a 



2^1(1 



1 + 2^(1-6) 



(35) 



In this case, the exponent coincides with the exponent given 
in Eq.(l33ll. Otherwise, (9 cr i t < Ogv), the minimum occurs at 
9 = Ogv- Therefore, we have 



lim — logE K%i (1 _ R) JB] 



-0 GF log(2Ve(l-e)). 



(36) 

if 9 cr u < Ogv- This exponent corresponds to the usual 
expurgated exponent for the BSC case (see also the discussion 
in [3]). The next corollary states the ACR of the upper bound 
of ML error probability in the case of 9 cr u < Ogv'- 

Corollary 3: If cr n < Ogv, the ACR is given by r] = 0. 
(Proof) Since the expurgated ensemble can be obtained from 
the original ensemble by removing a sub-exponential number 
of matrices, the exponent of the variance, i.e., 7(^1, O2), takes 
the same values for the original and expurgated ensembles if 



'GV 



< 0\,9-2, < 1 — 9gv- From Theorem [2] we have 



ri = max 

e GV <e<i-e GX 



— max 
e GV <8<i-9 GV 



H(9) + 29\og(2y/e(l-e)) 

l-R 



2H{6) + 26»log(2 v /e(l-e)) 



because q(9) — —00 for 9 < 9qv in the case of the 
expurgated ensemble. From the assumption 9 cr u < Ogv, 
2H(9) + 26»log(2 v /e(T — e)) is maximized at = G v- 
Note that -H(0 G y ) + l-R = holds. Moreover, H(0) + 
2#log(2i/e(l — e)) is also maximized at = Ogv- □ 

Appendix 

1) Preparation of the proof of Theorem Q} The second 
moment of the weight distribution for a given ensemble Q 
is given by 

= E e E E I[Hx = m ,Hy = m ] 
= E E Vg{I{Hx = m ,Hy = m }}. (37) 
For the case in which Q = TZ n m , we obtain 



Bn„, m 

- E E 



#{H : Hx = m ,Hy = m } 

Qrnn 



(38) 



Here, we encounter a problem of counting the matrices that 
satisfy both Hx = m and Hy = m . Before solving this 
counting problem, we first introduce some notation. 

Suppose that wi > and W2 > 0. For a given pair 

{x,y) 6 Z { - n ' W1 ^ x Z {n ^ W2 \ the index sets h, I 2 , h, and 
I4 are defined as follows: 1% = {k G : %k = l)2/fe = 

0},/ 2 = {fc G [l,n] : = l,y k = l},/ 3 - {fee [l,n] : 

Xk = 0,Vk = l},h = {k e [l,n] : x k = 0,y k = 0}, where 
x = {xx,x2, ■ ..,x n ) and y = (2/1,2/2, ■ ■ • ,y n )- The size of 
each index set is denoted by i k — #/fe(fc = 1,2,3,4). Let 
h = (hi, fi2, ■ ■ ■ , h n Y be a binary n-tuple (a row vector). 
The partial weight of h corresponding to an index set Ik(k = 

1,2,3,4) is denoted by Wh(h), namely, uik(h) = #{j G Ik : 
hj = l}. 

Since the index sets are mutually exclusive, the equation 
ii + *2 + *3 + H = n holds and i 2 can take the integer 
values in the following range: max{u>i + W2 — n, 0} < i 2 < 
min{wi, w 2 }- The size of each index set can be expressed as 
il = Wi — i 2 , i 3 = w 2 - 12, i± = n - (wi + 1V2 - 12)- 

The next lemma forms the basis for the proof of Theorem 

m 

Lemma 2: For any x G Z^ 1 ^ and y G Z(™^ 2 )(0 < 
w\,W2 < n), the following equalities hold: 



(39) 



(Proof) In the following, we prove the lemma for the condi- 
tions < wi < W2 < n. The proof for the opposite case 
< w 2 < wi < n then follows immediately upon exchanging 
the variables W2 and w\ in the proof. 
First, we will show that 



4{h G F 2 n : hx = 0, hy = 0} = 2 n ~ 



(40) 



if < wi < W2 < n and x ^ y. Let the support sets of 
x and y be S(x) — {i G [l,n] : Xi — 1} and S(y) ={iE 
[1, n] : yi — 1}, respectively. The following three cases should 
be treated separately: 

• Case (i): < 12 < wi (i.e., S(x) and S(y) overlap but 
S(y) does not include S(x).) 

• Case (ii): 12 = (i.e., S(x) and S(y) do not overlap.) 

• Case (iii): i 2 = u>i(i.e., S(y) includes S(x).) 

First, we consider Case (i). From the assumption that < 
i 2 < wi, it is clear that I\ ^ (because 12 < wi), I 2 ^ 
(because 12 > 0), I3 ^ $ (because W2 > wi > 12)- For any 
h G F2, the equations hx* = and hy* — hold if and only 
if Wi(h) is even for i = 1, 2, 3 or Wi(h) is odd for i = 1,2, 3. 
Thus, the number of vectors satisfying the above condition is 
given by 



N h = 2x 2 41 " 1 x 2 42 " 1 x 2 43 " 1 x 2 U = 2™~ 2 , 



where N h is defined by N h = G F 2 ™ : hx* = 0, hy* = 
0}. In the above derivation, we used the equalities: wi = 
ii + i 2 , W2 = «2 + h, U — n - + w 2 - «2)- Note that 
Eq. (ETj (and Eqs. (02] )(|43j, and (04]l to be presented below) 
holds regardless of the size of ^4(14 = or 14 > 0). 



(41) 
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We now consider Case (ii). For this case, I\ ^ (since 
Wi > 0), I2 = (since «2 = 0) and I3 ^ 9> (since w 2 > 
0). The equalities hx = and hy = hold if and only 
if Wi(h) is even for i = 1,3 holds. The number of vectors 
satisfying the condition is given by 

1—1 x 2*3~ 1 x 2* 4 = 2"~ 2 



N, 



h 



(42) 



The final case is Case (iii). For this case, 1\ = (since 
12 = Wi), I2 7^ (since 22 = wi > 0) and I3 / I (since 
x y and ?«i < 102). These conditions lead to the condition: 
Wi(h) is even for i = 2, 3 for faa; = 0, hy = 0. Again, 2™ -2 
n-tuples satisfy the above condition, namely, 

2 ~ 1 x 2 i3_1 x 2 U = 2 n ~ 2 . 



N. 



h 



(43) 



Combining the above results for Cases (i), (ii), and (iii), we 
obtain Eq. ( l40b . 

We then show that = 2 n ~ 1 holds if < wi = w 2 < n 
and x = y. For this case, we have Ii = 0, 1% 7^ 0, 73 = 0(since 
a; = y). Thus, the equations = 0, hy = hold if and only 
if W2(h) is even. The number of n-tuples satisfying the above 
condition is given by 

2*2-1 x = 2™- 1 



N, 



h 



(44) 



The proof of this lemma is completed. rj 
2) Proof of Theorem [7} The proof of Theorem Q] consists 
of two parts. The first part corresponds to the case in which 
the covariance becomes zero. The second part corresponds to 
the case in which the covariance becomes non-zero. 

We commence with the first part of the proof. Assume that 
< wi, W2 < n,x ^ y. From Lemma [2] we obtain 



#{H : Hx 



-- m ,Hy = O" 1 } 

m 

[] #{h e F? : hx 



0,hy = 0} 



k=l 

2?n(n — 2) 



(45) 



Substituting into 



A A 1 



we obtain 

= E 

n — 2m 



U'l 



E 

yez("- 
( n 

\W2 



2?n(n— 2) 



E 



[A W2 ]■ 
t [-^4^71 ) -A W2 ] 



(46) 
= 0. 



The last equality is equivalent to COV^ 

We now consider the second part of the proof: Assume that 
x = y. From Lemma [2] we have : Hx 1 = 0,iiy* = 

0} = 2 m< - n - 1 \ and 



E 



E 

xez(»- 

E 



E 
E 

■) yez(™- 



I[x = y]2 m ("- 1 ) 



I[x ^ y]2 m (™- 2 ) 



= 2~ 



= E- 



- 2- 



.(47) 



The last equality is equivalent to COVu n m {A Wl A w ) = (1 — 



*)2" 



□ 
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